Repository navigation
[Bugfix][Frontend] Reject an empty conversation in every renderer - #62218
albertoperdomo2 wants to merge 6 commits into
Conversation
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| # the HF tokenizer refuses an empty conversation in apply_chat_template, | ||
| # but renderers with their own chat encoding render it to a bare BOS | ||
| # prompt. Refuse it here so every renderer answers the same way. | ||
| if any(not conversation for conversation in conversations): |
There was a problem hiding this comment.
I don't think this is the right fix. As conversations grows larger, we should avoid introducing an additional loop.
There was a problem hiding this comment.
thanks for looking @chaunceyjiang! conversations here is the batch of conversations, not the messages in one, so the check is one O(1) emptiness test per conversation and it short-circuits
online serving always renders a single conversation (online_renderer.py:805), so there it's one check; only offline LLM.chat passes a batch
measured: 0.14µs for one conversation of any length, 16µs for 1000 conversations, which can be considered negligible next to rendering and tokenizing each of them IMHO
|
One more path this doesn't reach: Harmony (gpt-oss) chat completions. A check at the top of if not request.messages:
return self.create_error_response(
"messages must contain at least one message", param="messages"
)Happy to leave it to you to fold in here rather than open a separate PR. (Found while auditing chat-completion request validation against the OpenAI spec, which has |
Overview
Renderers with their own chat encoding (DeepSeek V3.2/V4/V4.1) render an empty conversation to a bare BOS prompt, so
/v1/responseswithinput: []returns 200 with text generated from nothing. This returns the same 400 that HF-tokenizer models already return.Claims
BadRequestError("Cannot apply chat template to an empty conversation. Provide at least one message.") for every renderer.input: []plusinstructionsorprevious_response_id.Validation
New
tests/renderers/test_empty_conversation.py: the 3 rejection cases fail onmainand pass with this change; the 2 controls pass on both.Details
transformersrefuses an empty conversation inapply_chat_template, but the DeepSeek tokenizers override that method and skip the check. The check now runs at the start ofBaseRenderer.render_chatandrender_chat_async, after each endpoint has built its messages. The Rust frontend already rejects an empty conversation (ChatRequest::validate); this brings the Python frontend in line.Claude Code used to create the new tests.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.